A psychometric assessment of a novel scale for evaluating vaccination attitudes amidst a major public health crisis

Despite abundant scientific evidence supporting immunization benefits, vaccine hesitancy remains a significant global health concern, particularly during public health crises. Exploring public attitudes towards vaccination is crucial. This study aimed to develop and validate a tailored Public Vaccination Attitudes Scale specifically under the unique circumstances of a public health crisis. A psychometric evaluation was conducted using a cross-sectional study during the peak of a major public health crisis. The scale was developed and its psychometric properties validated using three approaches: (1) generating the item pool through literature research and focus group discussions; (2) assessing the items through expert consultation; and (3) evaluating construct validity, content validity, and internal consistency reliability through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Data from a total of 3921 respondents were randomly divided into two subsets, one for EFA (n = 1935) and the other for CFA (n = 1986). A 22-item draft scale with five factors was created after literature research and focus group discussion. The content validity of this scale ranged between 0.88 and 1.00. EFA showed a 17-item scale with four factors (Cronbach’s α > 0.7) accounting for 68.044% of the total variance. CFA showed that the values of the fit indices, including convergent validity and discriminant validity, were excellent or acceptable. The overall Cronbach’s α was 0.874, and each factor ranged from 0.726 to 0.885. This study introduces a valuable tool for assessing vaccination attitudes during public health crises, aiding researchers, policymakers, and nurses in combating vaccine hesitancy. Emphasizing the importance of fostering vaccine acceptance, it enhances disease control during emergencies, contributing to the knowledge needed for more effective public health strategies and crisis responses


Expert consultation
Item evaluation was performed by the expert consultation method.Inclusion criteria were as follows: ① Expertise in hospital management, disaster management, public health management, vaccine research, and participation in public health prevention and control; ② greater than 5 years of professional experience; ③ bachelor's degree, associate senior title, or above; and ④ voluntary participation.Experts who could not be contacted electronically were excluded.A draft scale was sent to 20 experts at this stage, with 18 experts responding.If no response was received within 2 weeks, we considered it a lack of time or interest and discontinued contact.The experts classified each item of the scale as 'unimportant' , 'important but needs correction' , and 'important' .
The consensus coefficients for the two rounds were 90% and 93.3% respectively.All I-CVI data were greater than 0.88 28 .The experts' active participation and efficiency in both rounds demonstrate their keen interest in the study and reflect heightened global awareness and concern stemming from significant public health events like COVID-19.Vaccine acceptance plays a crucial role in safeguarding global health, thus making research on vaccination attitudes a prominent focus and trend.
In the first round of expert consultation, one suggestion was added: "Since the vaccine is free, I'm willing to take it." Additionally, five recommendations were modified.No items were added to or removed from the scale, but eleven items underwent corrections to rectify grammatical and spelling errors and enhance statement clarity.After the modifications, Kendall's W coefficient was 0.136 (P < 0.05), S-CVI/UA was 0.66, and S-CVI/Ave was 0.97.Following these revisions, we proceeded with the second round of expert consultation.The Kendall's W coefficient for the second round of Delphi consultation was 0.171 (P < 0.001).S-CVI/UA was 0.87, and S-CVI/ Ave was 0.99.The 22-item scale was created according to expert suggestions.All indicators showed that the scale exhibited relatively satisfactory content validity 29 and was recognized by experts from a professional point of view.

Factor analysis
EFA and CFA were performed to determine the construct validity together with the reliability of the scale, constituting the psychometric evaluation.
Item analysis and homogeneity tests were used to filter the initial items.This study adopted the extreme group method to analyse items.The difference between the top 27% and the bottom 27% of the total score of the questionnaire was compared by t tests.If P > 0.05, the item was deleted.We compared the high group (top 27%, total score ≥ 155 points) with the low group (bottom 27%, total score ≤ 82 points), and all items achieved significant P values (P < 0.001).In the homogeneity test, the correlation coefficient (R value) between items and the whole questionnaire was higher than 0.4, indicating that the acceptance attitudes measurement tool developed in this study exhibited a high level of homogeneity.
Kaiser-Meyer-Olkin (KMO) and Bartlett's Test of Sphericity were performed to assess the suitability of the questionnaire for exploratory factor analysis.A KMO value > 0.80 or a P value of Bartlett's Test < 0.05 indicates adequate sample size and suitability of variables for factor analysis.A further factor structure was developed with factors whose Cronbach's α was > 0.7 and item loading was > 0.4, which were then selected as the new structure.Confirmatory factor analysis (CFA) was used to test whether this factor structure was suitable, and convergent validity and discriminant validity were explored.Cronbach's α coefficient was used to assess the reliability of the scale.

Study design
The research adopted a cross-sectional design and adhered to the STROBE cross-sectional reporting guidelines 25 .Participation in the study was voluntary and anonymous, with informed consent obtained and documented via electronic signature.All collected data were treated confidentially and utilized solely for research purposes.

Data collection
Data collection occurred from June to August 2021, facilitated by a Sojump online survey, accessible through the Questionnaire Star survey website, the largest online survey platform in China, catering to research institutions' online questionnaire design and survey needs.Employing the self-selection online survey method, a nonprobability sampling approach 26 , participants were recruited via social network links.The study's target population comprised adults aged 18 years or older residing in mainland China.

Quality control
Throughout the investigation, two researchers were tasked with overseeing the completion of questionnaires and gathering adjustment recommendations via the website.Subsequently, upon concluding the investigation, we meticulously screened the data to ensure its accuracy.

Floor and ceiling effects
Among the participants, 8% scored the lowest possible score for the survey, and 5% scored the highest possible score.We introduced reverse items to address ceiling or floor effects.Additionally, the diverse participant pool and random data division into two parts for exploratory and confirmatory factor analyses also helped alleviate these effects to some extent.

Participants
A total of 4021 respondents participated in the survey using the Questionnaire Star survey website.However, 100/4021 (2.5%) of the questionnaires were removed from the sample according to the following exclusion criteria: (a) the surveys were incomplete; (b) the questionnaires were completed by respondents under 18 years of age; (c) the respondents did not consent using the first two questions; and (d) the same response options were chosen consistently.Ultimately, 3921 eligible surveys were analysed.

Statistical analysis
Data were analysed by IBM SPSS 20.0 for Windows and AMOS, and P < 0.05 was considered statistically significant in two-tailed tests.The general characteristics were described by means (SD) or frequencies for categorical items.Concordance among the experts was assessed using Kendall's W analysis.The structural validity of the scale was assessed using EFA and CFA.Cronbach's alpha was used for reliability analysis.

Ethics approval and consent to participate
The West China Hospital of Sichuan University Biomedical Research Ethics Committee approved the study (ethics number: 2021-992).All participants gave electronic informed consent for the participation.The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and the Helsinki Declaration of 1975, as revised in 2008.Participation in the study was anonymous.

Demographic characteristics
The study ensured the representativeness of survey participants by encompassing all adults aged 18 years and above nationwide.Participants had a mean age of 32.58 years (SD = 11.85;range = 18-85).Among the respondents, 77.5% were female (n = 3073), and 94.8% were ethnically Han (n = 3716).The highest educational level of 28.1% was a junior college degree (n = 1100).Almost half were married (51.3%).A total of 75.8% had a nonmedical background (2971), and 50.1% reported that their families had a medical background (1966).A total of 31.9% came from a large city, and 31.5% were from a mid-sized city.A total of 29.6% had an average family income (CNY) ≤ 5000 (Table 1).

Item analysis and item homogeneity test
The questionnaire was randomly divided into two parts, one for EFA (n = 1935) and the other for CFA (n = 1986).The results of item analysis and item homogeneity were based on EFA.All items achieved significant P values (P < 0.001).In the homogeneity test, the correlation coefficient (R value) between items and the whole questionnaire was higher than 0.4; the value for Item 10 was 0.395, so it was deleted.The remaining 21 items were included in the EFA (Table 2).

Structural validity and reliability
Principal component analysis and the promax method were used in EFA.Factors with eigenvalues above one were selected, while items with a maximum factor loading value < 0.4 and those with double loading were deleted if the factor load value of the main item was at least 2 times that of the minor factor as the entry inclusion criterion.
The KMO value was 0.888, and Bartlett's test of sphericity was significant (X 2 = 17999.208,df = 136, P < 0.001), the data were ideal for factor analysis.The cumulative contribution rate was 68.044% indicated satisfactory construct validity.
A new four-factor structure was formed in Table 3.The Cronbach's α of all factors was above 0.874, and the coefficients of recognition, environment, value and safety were 0.885, 0.839, 0.882 and 0.726, respectively.The CFA results were as follows: CMIN/df = 7.518, RMR = 0.042, GFI = 0.9530, AGFI = 0.0.935,NFI = 0.958, RFI = 0.949, IFI = 0.964, CFI = 0.964 and RMSEA = 0.056 (Fig. 1, Tables 4 and 5).All values of GFI, AGFI, NFI, RFI, IFI, TFI and CFI were above 0.9 30 .CMIN/df was greater than 3, RMSEA was less than 0.08, and RMR was less than 0.05, which indicated that the four-factor structure fit excellently 31 .When the sample size exceeds 1000, the chi-square value of the model will also increase, so CMIN/DF was not strictly considered 32 .All values of composite reliability (CR) of the four dimensions were greater than 0.7 33 , which means they achieved the required level.The values of average variance extracted (AVE) in Factors 2 and 4 were below but close to 0.5, indicating adequate construct reliability and adequate convergent validity 34 .For Factors 2 and 4, although the AVE value was less than 0.5, the other indicators were satisfactory; the CR values of the two factors were 0.835 and 0.747, respectively.Therefore, the factors were kept in the acceptance attitudes to vaccines scale.The square between the respective constructs was less than the square of AVE for the construct that achieved the discriminant validity of the acceptance attitudes towards vaccines.

Discussion
In this study, we created a scale for evaluating acceptance attitudes towards vaccination amidst a major public health crisis.A 17-item scale with four factors was confirmed: recognition (5 items), environment (6 items), value (3 items) and safety (3 items) formed the public acceptance attitudes towards vaccination scale.
Factor one was named 'recognition' and included 5 items (A15/A16/A17A18/A19).The Cronbach's α value was 0.885, and item loading ranged from 0.528 to 0.898.They were all recommended and approved by authoritative institutions or trusted individuals, including the country, work units, medical staff, and family/friends.The www.nature.com/scientificreports/price of vaccines plays a significant role in determining their affordability for individuals and serves as a primary factor in assessing vaccine acceptance.Recognition facilitates the rapid identification of factors influencing vaccine confidence and the establishment of manageable response strategies during public health emergencies.
Confidence is a key influencing factor in vaccine hesitancy 35 , and recommendations from trusted individuals in the community can enhance individuals' confidence in vaccine acceptance.A research reveals that 87.09% of respondents deemed their doctor's recommendation as crucial for decisionmaking 36 .Recognition also significantly influenced individuals' vaccination attitudes, serving as a cornerstone in fostering confidence and trust in the vaccination process.This contributed to increased vaccine acceptance amid major public health crises, as individuals prioritize safeguarding themselves and others from disease risks.However, the sudden and unpredictable nature of such crises can lead to fluctuations in vaccine acceptance.This is attributed to asymmetrical information regarding vaccines, where initial misinformation or incomplete knowledge may foster misunderstandings and resistance towards vaccination, fueling panic and distrust 37 .Yet,  Factor 2 was named 'environment' and included 6 items (A5/A6/A8/A9/A13/A14).The Cronbach's α value was 0.839, and item loading ranged from 0.491 to 0.756.This factor represents the perception of the current risk of infectious disease outbreaks; if individuals perceive risk, they will behave to mitigate the risk.Vaccination is the most important response measure.Accordingly, people will exhibit positive response behaviour; otherwise, their behaviour involves hesitation and a lack of acceptance of the vaccine.A global survey indicates that individuals' attitudes towards vaccine acceptance have been influenced by the information disseminated through various social media platforms and the severity of COVID-19 38 .Environmental experience is a crucial external factor determining individual vaccine acceptance 39 .According to surveys, there is a significant contrast in individuals' attitudes towards vaccine acceptance during and after the COVID-19 pandemic 38,40 .When individuals are experiencing a major public health crisis like COVID-19, their perception of the imminent threat to life and health is deeper and more genuine, providing a more objective reflection of public attitudes towards vaccine acceptance or refusal.This differs from conventional factors influencing vaccine acceptance attitudes.However, after this period, vaccine acceptance attitudes are influenced by various sources of information, such as media reports and individual experiences.Therefore, the measurement of the environmental dimension in this study provides a more authentic and objective reflection of the current situation.
Factor 3 was named 'value' and included 3 items (A1/A2/A3).The Cronbach's α value was 0.882, and item loading ranged from 0.781 to 0.8906.This factor captured opinions about the role and value of the vaccine.Whether an individual accepts a vaccine depends mainly on whether the vaccine can play a role in reducing infection.If the vaccine can reduce the infection rate and the rate of serious disease after infection, then the vaccine plays a role and can be positively recognized and accepted.The value dimension acts as a direct determinant of individual vaccine acceptance attitudes and also indirectly reflects individuals' health literacy regarding vaccines.Higher levels of health literacy regarding vaccines indicate a deeper understanding of the role and importance of vaccines in disease prevention.Conversely, lower levels of perceived value may signify knowledge gaps or misconceptions about vaccines, resulting in hesitancy or reluctance to accept vaccination.Understanding and addressing the factors influencing perceptions of vaccine value are essential for promoting vaccine acceptance and effectively addressing vaccine hesitancy 41 .
Factor 4 was named 'safety' and included 3 items (A4/A12/A20).The Cronbach's α value was 0.726, and item loading ranged from 0.600 to 0.789.The three items were "quality and safety", "protective effect", "quality of the vaccine produced by the manufacturer" and all involved the theme of security.The most important consideration for an individual when getting vaccinated is the safety of the vaccine.The safety and efficacy of the vaccine are reasons for an individual to overcome hesitation, which is also in line with medical decision-making behaviour theory 42 .Safety is a key factor shaping individuals' vaccine acceptance, especially during unforeseeable public health crises.When individuals are unsure about vaccine efficacy, safety becomes their primary concern 43 .Hence, strategies addressing safety concerns and providing transparent, accurate vaccine safety information are crucial for improving vaccination attitudes and uptake rates.The scale is a unique and practical tool for assessing vaccination attitudes during a major public health crisis, providing distinct advantages and practical significance.Although a multitude of tools exist for assessing vaccine acceptance attitudes, including those tailored to COVID-19 vaccines, and extensive research has examined diverse predictive factors such as gender, age, education level, and economic development [35][36][37][38][39][40][41] , the emphasis of these investigations predominantly centers on uncontrollable variables.Addressing uncontrollable factors poses challenges for short-term resolution.During major public health crises, there is an urgent imperative to promptly tackle controllable factors that impact vaccine acceptance attitudes.The four dimensions of our research instrument are well-suited for evaluating vaccine acceptance attitudes amidst major public health crises.Recognition, environment, safety and value were the main factors in vaccine acceptance decision-making.Whether an individual makes a decision depends on both intrinsic and extrinsic factors centred on vaccine safety.Internal factors include vaccine safety and value, namely, whether the vaccine plays a reliable role.External factors include the urgency of the environment and the degree of recognition of the vaccine.Internal and external factors also involve weighing the pros and cons of vaccine acceptance or refusal.This can provide not only factors for public vaccine acceptance decision-making but also the weight of the factors to provide a basis for screening and decision-making for vaccine acceptance in different environmental settings.
The initial 5 items (A7/A10/A11/A21/A22/), which were about responses to individual knowledge of vaccines, were deleted and excluded from statistical analysis based on numerical exclusion criteria.In addition to statistical reasons, we analysed other possibilities.One was based on the realistic principle that individuals care more about the function and safety of vaccines than they do about the knowledge of vaccines.Previous studies have considered vaccine knowledge to be important 36 , but for people from non-medical backgrounds, especially in emergency settings, the safety and value of vaccines are considered more important.The other possibility is that individuals are very knowledgeable about vaccines because vaccines have been used successfully against many viruses throughout history.Finally, a statistical descriptive summary of several items showed that the scores ranged from 3.93 ± 0.88 to 4.66 ± 0.66, which were all relatively high.This indicated that knowledge of vaccines was not the main consideration for deciding whether to accept vaccines, but this specific explanation needs further research and analysis.

Limitations
This study has several limitations.Firstly, the use of an online survey platform, while convenient and efficient, may introduce selection bias and raise concerns about response accuracy and misinterpretation of questions.Additionally, the inability to verify respondent identity online may compromise data validity.However, it's worth noting that these limitations may not have significantly impacted the results.Secondly, the study exclusively utilized quantitative research methods to explore vaccination acceptance attitudes, suggesting a need for future research to incorporate qualitative approaches.Thirdly, retest reliability was not conducted, but it is planned for inclusion in a subsequent study.Lastly, the large sample size may potentially introduce ceiling or floor effects.To address this, we diversified sample sources and partitioned the dataset for exploratory and confirmatory factor analyses, aiming to mitigate these effects to some extent.

Conclusions
We developed a scale for measuring acceptance attitudes towards vaccination and validated it with a relatively adequate sample size.This scale is a 17-item scale with 4 factors and excellent reliability and validity, making it acceptable and useful.It provides an effective evaluation tool for assessing whether individuals have adopted vaccination behaviour.The scale serves as a rapid assessment tool for identifying public vaccination acceptance attitudes, particularly during major public health crises.It enables healthcare practitioners to recognize and comprehend the key factors influencing vaccine acceptance attitudes, facilitating targeted health education and promotional campaigns aimed at altering public attitudes and behaviors, thereby promoting vaccination and reducing hesitancy.The dimensions assessed by the scale represent critical and controllable factors that the public primarily considers during major public health crises.Consequently, it can guide health policymakers in promptly understanding public attitudes toward vaccine acceptance and formulating more targeted and effective vaccination policies to enhance vaccination rates across diverse demographic groups.Additionally, healthcare professionals can employ the scale periodically to assess dynamic trends in vaccine acceptance attitudes and adjust relevant promotional and educational strategies in a timely manner, ensuring the effective implementation of vaccination activities.We look forward to witnessing the widespread application of this scale in practice and its contribution to addressing current and future global health challenges.Meanwhile, the scale can be used as a necessary part of the emergency plan system for screening and decision-making.The four dimensions in the scale contribute to building a knowledge system and model of vaccine acceptance, thereby enhancing strategies for major public health crisis prevention and control.Finally, this scale can be widely applied and tested in practice, contributing to addressing current and future global health challenges. https://doi.org/10.1038/s41598-024-61028-zwww.nature.com/scientificreports/

4 A 20 I 682 Figure 1 .
Figure 1.Confirmatory factor analysis of the acceptance attitudes towards vaccination scale.

Table 1 .
Characteristics of the participants.

Table 3 .
Factors and item loadings explored by exploratory factor analysis.